Media encoding using changed regions

ABSTRACT

A method and system for media encoding using changed regions. The method includes detecting the changed regions and a static portion of a desktop of a computing device. The amount of changed regions may be determined. The changed regions may be encoded in response to the amount of changed regions.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.13/568,703 filed Aug. 7, 2012 entitled “Media Encoding Using ChangedRegions”, which is incorporated by reference herein in its entirety.

TECHNICAL FIELD

The present invention relates generally to media encoding. Morespecifically, the present invention relates to media encoding usingchanged regions.

BACKGROUND ART

Various uses of a computing device include scenarios with a portion ofthe display being static, while other portions of the display areactive. For example, wireless display (WiDi) is a technique by which adesktop of an electronic device is rendered on a remote display,wirelessly. For example, a tablet device may send all images on itsdesktop to a television to be rendered. Typical uses for WiDi mayinclude online video playback over a web browser and video chat. Each ofthese uses involve a portion of the display that is static, whileanother portion of the display may be dedicated to the video encodingover a web browser or video chat. In other examples, participating in avideo conference includes a portion of the display that is static, andanother portion of the display that is dedicated to rendering images ofother the video conferencing participants. In any event, the use of WiDiand video conferencing may consume a relatively large amount of power,as the images from the display to be rendered are typically encoded,decoded, and processed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a diagram of media encoding within a web browser, inaccordance with embodiments;

FIG. 1B is a diagram of media encoding of a video conference, inaccordance with embodiments;

FIG. 2 is a block diagram of a computing device that may be used toprovide media encoding using changed regions, in accordance withembodiments;

FIG. 3 is a schematic of a WiDi architecture that may be used to providemedia encoding using changed regions, in accordance with embodiments;

FIG. 4 is a block diagram showing a process flow of a media encodingusing changed regions, in accordance with embodiments;

FIG. 5 is a process flow diagram showing a method for media encodingusing changed regions, in accordance with embodiments;

FIG. 6A is a process flow diagram showing a method for media encodingusing changed regions, in accordance with embodiments;

FIG. 6B is a process flow diagram showing a method for media encodingusing changed regions, in accordance with embodiments; and

FIG. 7 is a block diagram showing tangible, non-transitory,computer-readable media that stores code for media encoding usingchanged regions, in accordance with embodiments;

FIG. 8 is a block diagram of an exemplary system for implementing sharedphysical memory; and

FIG. 9 is a schematic of a small form factor device in which the systemof FIG. 8 may be embodied.

The same numbers are used throughout the disclosure and the figures toreference like components and features. Numbers in the 100 series referto features originally found in FIG. 1; numbers in the 200 series referto features originally found in FIG. 2; and so on.

DESCRIPTION OF THE EMBODIMENTS

As discussed above, WiDi is a technique by which the images on a displayof a computing device are wirelessly rendered on a remote display. Thedisplay of a computing device may be referred to as a desktop, and caninclude several icons and applications in various positions on thedesktop. Traditional implementations of WiDi encode the entire contentsof the desktop, wirelessly send the encoded desktop to another device,then decode and process the entire desktop in order to render thedesktop on another display, such as a television or computer monitor.Processing the entire desktop using WiDi techniques can consume a largeamount of power. Additionally, processing the entire desktop usingtraditional WiDi techniques can consume a large amount of graphicsprocessing unit (GPU) time and a large portion of network bandwidth.Similarly, video conferences may be rendered on a computing device bydecoding, encoding, and processing a data stream composed of the entiredesktop. Processing the entire desktop of a video conference can consumea large amount of GPU time and a large portion of network bandwidth.Both GPU bottlenecks and a lack of network bandwidth may contribute tolatency in rendering the desktop during a video conference or a WiDisession. Accordingly, embodiments described herein relate to mediaencoding using changed regions. The dynamic content of the media may besent to a display for rendering along with changed region positionvalues.

The use of changed regions is an image rendering technique where thedesktop is divided into several regions. As used herein, a region refersto an area of the desktop, of any size or shape. For example, a regionmay be in the shape of a rectangle. Regions may be obtained from thedesktop at various intervals. A region may be a changed region when oneor more pixels within the region have changed when compared with aprevious region that is the same size and shape as the region, and is inthe same position on the desktop as the region. When the region is arectangle, and the rectangle contains any changed pixels, it may bereferred to as a dirty rectangle. In embodiments, each changed regionmay be written to a buffer in order to be composed with a static portionof the desktop and rendered.

In embodiments, a change in a pre-determined number of pixels mayindicate a changed region. A changed pixel is a change in any value ofthe pixel, including but not limited to, address, intensity, color, orany combination thereof. The pre-determined number of pixels may be athreshold for determining a changed region. The threshold may be anynumber of pixels. For example, a threshold may indicate that ten percentor more of the pixels within a region should change in order for theregion to be a changed region. Accordingly, if less than ten percent ofthe pixels with the region change, then the region may not be considereda changed region. In this manner, a changed region may be detected whena number of changed pixels within a region that have changed is greaterthan the predetermined threshold. For example, the threshold may belower in applications where premium graphics are displayed, such as highdefinition playback. When high definition playback is performed, thethreshold may be selected in a manner such that high definition playbackcan be achieved while encoding the playback using changed regions.Additionally, in embodiments where power savings is a concern, thethreshold may be set in a manner that enables a device to conserve powerwhile encoding a playback using changed regions.

FIG. 1A is a diagram of media encoding within a web browser 100, inaccordance with embodiments. The web browser 100 includes an area 102that renders a video selected by a user. The area 102 could be a singleregion or the area 102 could include several regions, as describedabove. The remaining components of the web browser 100 are typicallyfixed while the area 102 is playing a video selected by a user.Accordingly, links to other videos within an area 104 typically remainunchanged while area 102 is changing due to media playback. Further, thearticle in area 106 below the area 102 remains unchanged while media isbeing played within the area 102. As used herein, media playback refersto any dynamic content, such as online video broadcasts or videoconferencing images. Additionally, similar to the area 102, theremaining components of the web browser 100 may form a single region orseveral regions of the web browser 100. In embodiments, the region mayinclude contiguous or non-contiguous pixels.

Generally, media playback on a computing device includes an area, suchas area 102, that is dedicated to media playback, and thus changes overtime and is dynamic. For example, a video chat application enables alocal user to talk with a remote user using a camera and microphone of acomputing device. The area that renders the images or video of theremote user to the local user is constantly changing in response to thevarious positions of the remote user. The remainder of the display,which can include a background of the computing device as well as otherportions of the video chat application, are typically static. In anotherexample, a presentation application can enable one or more users toconnect to an electronic presentation. During the electronicpresentation, one or more users may interact with slides that arerendered during the presentation. The area that renders images of theslides is constantly changing, while the remainder of the display isstatic.

FIG. 1B is a diagram of media encoding of a desktop 110 with a videoconference, in accordance with embodiments. The desktop 110 includes anarea 112 that renders documents, slides, images, and videos presentedduring the video conference. The area 112 could be a single region orthe area 112 could include several regions. The remaining components ofthe desktop 110 are typically fixed while the area 112 changes inresponse to the information presented during the video conference. Forexample, the icon at reference number 114 is generally static during thevideo conference presented within the area 112. The remaining componentsof the desktop 110 may form a single region or several regions of thedesktop 110.

In the following description and claims, the terms “coupled” and“connected,” along with their derivatives, may be used. It should beunderstood that these terms are not intended as synonyms for each other.Rather, in particular embodiments, “connected” may be used to indicatethat two or more elements are in direct physical or electrical contactwith each other. “Coupled” may mean that two or more elements are indirect physical or electrical contact. However, “coupled” may also meanthat two or more elements are not in direct contact with each other, butyet still co-operate or interact with each other.

Some embodiments may be implemented in one or a combination of hardware,firmware, and software. Some embodiments may also be implemented asinstructions stored on a machine-readable medium, which may be read andexecuted by a computing platform to perform the operations describedherein. A machine-readable medium may include any mechanism for storingor transmitting information in a form readable by a machine, e.g., acomputer. For example, a machine-readable medium may include read onlymemory (ROM); random access memory (RAM); magnetic disk storage media;optical storage media; flash memory devices; or electrical, optical,acoustical or other form of propagated signals, e.g., carrier waves,infrared signals, digital signals, or the interfaces that transmitand/or receive signals, among others.

An embodiment is an implementation or example. Reference in thespecification to “an embodiment,” “one embodiment,” “some embodiments,”“various embodiments,” or “other embodiments” means that a particularfeature, structure, or characteristic described in connection with theembodiments is included in at least some embodiments, but notnecessarily all embodiments, of the inventions. The various appearancesof “an embodiment,” “one embodiment,” or “some embodiments” are notnecessarily all referring to the same embodiments. Elements or aspectsfrom an embodiment can be combined with elements or aspects of anotherembodiment.

Not all components, features, structures, characteristics, etc.described and illustrated herein need be included in a particularembodiment or embodiments. If the specification states a component,feature, structure, or characteristic “may”, “might”, “can” or “could”be included, for example, that particular component, feature, structure,or characteristic is not required to be included. If the specificationor claim refers to “a” or “an” element, that does not mean there is onlyone of the element. If the specification or claims refer to “anadditional” element, that does not preclude there being more than one ofthe additional element.

It is to be noted that, although some embodiments have been described inreference to particular implementations, other implementations arepossible according to some embodiments. Additionally, the arrangementand/or order of circuit elements or other features illustrated in thedrawings and/or described herein need not be arranged in the particularway illustrated and described. Many other arrangements are possibleaccording to some embodiments.

In each system shown in a figure, the elements in some cases may eachhave a same reference number or a different reference number to suggestthat the elements represented could be different and/or similar.However, an element may be flexible enough to have differentimplementations and work with some or all of the systems shown ordescribed herein. The various elements shown in the figures may be thesame or different. Which one is referred to as a first element and whichis called a second element is arbitrary.

FIG. 2 is a block diagram of a computing device 200 that may be used toprovide media encoding using changed regions, in accordance withembodiments. The computing device 200 may be, for example, a laptopcomputer, desktop computer, tablet computer, mobile device, server, orcellular phone, among others. The computing device 200 may include acentral processing unit (CPU) 202 that is configured to execute storedinstructions, as well as a memory device 204 that stores instructionsthat are executable by the CPU 202. The CPU 202 can be a single coreprocessor, a multi-core processor, a computing cluster, or any number ofother configurations. Furthermore, the computing device 200 may includemore than one CPU 202. The instructions that are executed by the CPU 202may be used to implement media encoding using changed regions. Thememory device 204 can include random access memory (RAM), read onlymemory (ROM), flash memory, or any other suitable memory systems. Forexample, the memory device 204 may include dynamic random access memory(DRAM).

The computing device 200 may also include a graphics processing unit(GPU) 206. As shown, the CPU 202 may be connected through a bus 208 tothe GPU 206. The GPU 206 may be configured to perform any number ofgraphics operations within the computing device 200. For example, theGPU 206 may be configured to render or manipulate graphics images,graphics frames, videos, or the like, to be displayed to a user of thecomputing device 200. The GPU 206 may also include one or more engines,such as the engine 210 and engine 212. An engine is a component of a GPUthat may be used for parallel processing of data sent to the GPU.Although two GPU engines are shown, the GPU may contain any number ofengines. In embodiments, the engine 210 or the engine 212 is acomposition engine that may be used to compose the static portion of thedesktop with the dynamic areas of the desktop during video conferencingor WiDi.

The computing device 200 may also include one or more cameras 214. Thecameras 214 may be used to capture video and images for various uses,such as video conferencing and video chat applications. Additionally,the computing device 200 includes a microphone 216 that may be used tocapture sound for various uses, such as a video conferencing and videochat applications. The CPU 202 may be connected through the bus 208 toother input/output (I/O) components using an I/O device interface 218configured to connect the computing device 200 to one or more I/Odevices 220. The I/O devices 220 may include, for example, a keyboardand a pointing device, wherein the pointing device may include atouchpad or a touchscreen, among others. The cameras 214, microphone216, and I/O devices 220 may be built-in components of the computingdevice 200, or may be devices that are externally connected to thecomputing device 200.

The CPU 202 may also be linked through the bus 208 to a displayinterface 222 configured to connect the computing device 200 to adisplay device 224. The display device 224 may include a display screenthat is a built-in component of the computing device 200. The displaydevice 224 may also include a computer monitor, television, orprojector, among others, that is externally connected to the computingdevice 200.

The memory device 204 may include one or more drivers. The driversenable a piece of hardware or an application to communicate with theoperating system, applications, or other hardware of the computingdevice 200. As used herein, the ability to communicate includes theconversion of data from one format to another, or any technique thatfacilitates the processing or transfer of data between hardware orsoftware components. For example, the computing device 200 may include agraphics driver 226, a media driver 228, and a WiFi driver 230. Thegraphics driver 226 enables the GPU to communicate with the operatingsystem and various applications, while the media driver 228 enables amedia application to communicate with the operating system and variousapplications. The WiFi driver 230 enables the operating system tocommunicate with hardware and applications responsible for wirelesstransmissions to and from computing device 200.

The computing device 200 may also include a storage 232. The storage 232is a physical memory such as a hard drive, an optical drive, athumbdrive, an array of drives, or any combinations thereof. The storage232 may also include remote storage drives. The storage may also includeone or more media applications 234. The media applications 234 include,but are not limited to, video playback applications, video chatapplications, video conferencing applications, and presentationapplications. In examples, the media driver 228 may include a timer thatis used to schedule intervals for detecting any changed regions from themedia application 234. Additionally, in examples, a media application234 may be polled by the media driver 228 in order to detect any changedregions from the media application 234 at various intervals.Furthermore, in examples, the changed regions are pushed to the mediadriver 228 by the media application 234 when they are detected atvarious intervals. The changed regions may also be pulled from the mediaapplication 234 by the media driver 228 when changed regions aredetected at various intervals.

The media playback from media applications 234 may be sent to a streammanager 236 along with the other contents of the desktop. The streammanager 236 may mange the encoding, decoding, and composition of thedesktop. In embodiments, the stream manager 226 may obtain positionvalues for each changed region.

The storage 232 may also include a middleware 238 and a WiDi application240. The middleware 238 may be used to encode the changed regions andcalculate position values of the changed regions. The WiDi applicationmay prepare the encoded changed regions and their position values to besent wirelessly to a remote adapter 242. The remote adapter may bephysically separated from the computing device 200. In embodiments, theremote adapter 242 is another computing device. A network interfacecontroller (NIC) 244 may be configured to connect the computing device200 to a network 246. Network 246 can be a wire line network, a wirelessnetwork, or a cellular network. The network 246 may be any wide areanetwork (WAN), any local area network (LAN), or the Internet, amongothers. For example, network 246 can be 3GPP LTE network or a WiFinetwork. In embodiments, a remote adapter 242 may obtain encoded changedregions from the computing device 200 using the NIC 244 and the network246.

The remote adapter 242 may contain a GPU 248. The GPU 248 may include acomposition engine 250. In embodiments, the composition engine 250 maycompose the changed regions with a static portion using the changedregion position values. The static portion includes the areas of thedesktop that do not include changed regions. The composed changedregions and static portion may be rendered on a remote device 252. Inembodiments, the remote adapter 242 and the remote device 252 are thesame component.

The block diagram of FIG. 2 is not intended to indicate that thecomputing device 200 is to include all of the components shown in FIG.2. Further, the computing device 200 may include any number ofadditional components not shown in FIG. 2, depending on the details ofthe specific implementation.

FIG. 3 is a schematic of a WiDi architecture 300 that may be used formedia encoding using changed regions, in accordance with embodiments.Like numbered items are as described with respect to FIG. 2. The WiDiarchitecture 300 includes hardware 302 that supports the media encodingusing changed regions. The hardware may include, but is not limited to,the CPU 202 and the GPU 206 (FIG. 2). The graphics driver 226 enablesthe hardware to communicate with other hardware and variousapplications. The hardware composer 304 or the desktop compositionengine 306, or any combination thereof, may determine which regions ofthe desktop have changed. In embodiments, the hardware composer 304 maybe a Hardware Composer that is a Hardware Abstraction Layer (HAL) withinthe Android Operating System by the Open Handset Alliance. Further, inembodiments, the desktop composition engine 306 may be a DesktopComposition Manager or a Desktop Window Manager of the Windows OperatingSystem by Microsoft. Moreover, in embodiments, various hardware such asa fixed function unit, video encoder, or a scene change detectionhardware unit may be used to determine the changed regions.

Accordingly, the hardware composer 304, the desktop composition engine306, fixed function unit, video encoder, or a scene change detectionhardware unit may be used to obtain the changed regions. In embodiments,the current regions of the desktop may be compared with precedingregions obtained from the desktop. The regions may be obtained throughpushing, pulling, polling, or sent to the hardware composer 304, thedesktop composition engine 306, a fixed function unit, a video encoder,or a scene change detection hardware unit from the WiDi stream manager236 at scheduled intervals as discussed above. If there is a changebetween a current region of the desktop and the same region previouslysent from the WiDi stream manager 236, the region has changed. Inembodiments, the region may also be compared to a predeterminedthreshold of changed pixels. If the number of pixels that have beenchanged are greater than the predetermined threshold, then the regionmay be considered a changed region. In response to detecting a changedregion, the hardware composer 304, the desktop composition engine 306, afixed function unit, a video encoder, or a scene change detectionhardware unit may send the changed region position values to the streammanager 236.

The WiDi stream manager 236 manages the composition and encodingperformed by the middleware 238. In embodiments, the WiDi stream managercommunicates with the middleware 238 to obtain the image information ofthe changed regions, which may include the pixels to encode. The imageinformation corresponds to the changed region position values for thechanged regions obtained via one or more of the hardware composer 304,desktop composition engine 306, a fixed function unit, a video encoder,a scene change detection hardware unit, or any combination thereof. Inembodiments, the image information of the changed regions form a videosurface. As used herein, the video surface is a dynamic portion of thedisplay that is used to render a video or other media on the desktop.

The middleware 238 can obtain the image information on the desktop fromthe media application 234 through the WiDi stream manager 236. Forexample, media application 234 may compress a video playback accordingto video standards such as those by the Moving Picture Experts Group(MPEG) or H.264. The MPEG and H.264 video standards may compress a videoby processing the video frame-by-frame and using various encodingalgorithms. Middleware 238 may decode the video using a decoder 308.

The middleware 238 may also include a video processor 310 that processesthe video as directed by the WiDi stream manager 236. Video processingmay include enhancing the decoded video by scaling the video or changingthe color space of the video. For example, the video may be scaled froma higher resolution for playback to a lower resolution for playback. Inembodiments, the media application 234 is a video conferencingapplication, and the middleware 238 includes a composer 312 thatcomposes the dynamic portion received from the video conferencingapplication with the static portion of the desktop. The composed dynamicportion and static portion may be rendered on the computing device, suchas computing device 200 (FIG. 2).

The middleware 238 may encode the changed regions using an encoder 314.In embodiments, the middleware 238 includes a multiplexer 316 thatcombines the image data with audio data. The middleware 238 may alsoprepare the changed regions for transmission to a remote adapter atblock 318 using a transmitter 318. The WiFi driver 230 may enablehardware such as the NIC 244 (FIG. 2) to transmit the encoded changedregions along with their changed region position values using themiddleware 238 and the WiDi application 240.

FIG. 4 is a block diagram 400 showing the data flow of media encodingusing changed regions, in accordance with embodiments. The client device402 contains the data flow that may occur using a computing device suchas the computing device 200 (FIG. 2), while the remote device 404contains the data flow that may occur using the remote adapter 242 orthe remote device 252 (FIG. 2). In embodiments, media encoding usingchanged regions occurs using a computing device without the use of WiDi.

The compressed video 406 may be obtained from a media encodingapplication, such as media application 234 (FIG. 2). The compressedvideo may be decoded at block 408. As discussed above, the compressedvideo may be decoded using middleware 228 (FIG. 2). The decoded videomay include both a video surface and background surface from the mediaencoding. As discussed above, the video surface is a dynamic portion ofthe desktop that changes over time. By contrast, a desktop surface is astatic portion the desktop that remains fixed over time. The videosurface 410 may be obtained by retrieving the changed regions frommiddleware 238 as described above with respect to FIG. 3.

The video surface 410 may be encoded at block 412. In embodiments, theencoded video surface is sent to a buffer 414, and then transmitted atblock 416 to a remote device 404. The remote device may decode the videosurface at block 418. The video surface 410 may be processed at 420 withthe static desktop surface 422. Video processing may include, but is notlimited to scaling, rotation, and de-noise of the video. The staticdesktop surface 422 includes regions that are not dirty and have beenpreviously transmitted to the remote device 404. The video surface 410and the static desktop surface 422 may be composed into one image atblock 424. The media encoding may be sent to a display 252 in order tobe rendered on the display.

FIG. 5 is a process flow diagram showing a method 500 for media encodingusing changed regions, in accordance with embodiments. In variousembodiments, the method 500 may be executed on a computing device, suchas the computing device 200. In other embodiments, the method 500 may beexecuted using a WiDi architecture such as the WiDi architecture 400discussed above with respect to FIG. 4.

At block 502, the changed regions and a static portion of the desktopare detected. As discussed herein, the changed regions may be detectedby a hardware composer 304 or desktop composition engine 306 (FIG. 3).In embodiments, the changed regions may be detected using a mediadriver. Furthermore, as discussed above, the changed regions may bedirty rectangles. At block 504, the amount of changed regions may bedetermined. The amount of changed regions may represent a threshold. Inembodiments, if the number of changed regions exceeds the threshold, theentire display should be composed and rendered. At block 506, thechanged regions of each frame is encoded. In embodiments, each changedregion and the corresponding changed region position values are sent toa composition engine, and encoded changed regions may be decoded. Thechanged regions may be composed with the static portion using theirchanged region position values. The composition engine may be located ona remote adapter, or the composition engine may be a component of thecomputing device that detects the changed regions. Furthermore, inembodiments, one or more changed regions and their changed regionposition values may be sent to another computing device using a network,such as the Internet.

FIGS. 6A and 6B are a process flow diagram showing a method 600 formedia encoding using changed regions, in accordance with embodiments. Invarious embodiments, the method 600 may be executed on a computingdevice, such as the computing device 200. In other embodiments, themethod 600 may be executed using a WiDi architecture such as the WiDiarchitecture 400 discussed above with respect to FIG. 4.

The method 600 begins at block 602 by determining if there are anychanged regions. In embodiments, the changed regions may be detected ina video surface of the media encoding or a desktop surface of the mediaencoding. If there are no changed regions, process flow remains at block602 until a changed region is detected. When a changed region within thevideo surface or the desktop surface occurs, process flow continues toblock 604.

At block 604, it is determined if more than n regions are dirty. Thevalue of n may be predetermined, and represents a threshold amount ofchanged regions. If more than n regions are dirty, that may indicatethat the user has resized a window or closed the media application. Inthe case where more than n regions are dirty, the entire display shouldbe composed and rendered. Accordingly, if more than n regions are dirty,then process flow continues to block 606. If less than n regions aredirty, process flow continues to block 610.

At block 606, the entire desktop is composed. In embodiments, changedregions of the desktop are detected by checking a back buffer. The backbuffer may include the image information associated with the staticportion. The changed regions of the desktop along with the changedregion positions may be sent to a remote adapter. The entire desktop maybe composed when the amount of changed regions that have changed aregreater than the threshold amount n. At block 608, the entire desktop isencoded using intra-coded frames (I-frames) or predictive picture frames(P-frames). When the desktop is encoded using I-frames, each individualframe is fully specified within the encoding. Thus, an I-frame conveysthe entire image information without use of data from the previousframes. Each I-frame can be thought of as a complete static image of themedia encoding. When the desktop is encoded using P-frames, the changesbetween the current frame and the previous frame are encoded. Theunchanged pixels of the image are not encoded, and the frame relies onsome image information from the previous frames when the frame isencoded.

At block 610, the changed region position values are calculated for eachchanged region. At block 612, it is determined if the position valuesfor each changed region of the current frame are identical to the regionposition values from the previous frame. If the position values of thechanged regions of the current frame matches the position values of theregions in the previous frame, then the changed regions of the currentframe can be encoded using either an I-frame or a P-frame. In thisscenario, either an I-frame or a P-frame may be used to encode thechanged regions of the current frame, as the each changed region has aprevious region in the previous frame that is located in the sameposition. As a result, the previous frame contains image information foreach region at each location. Accordingly, the changed regions of thecurrent frame can be fully specified, as is the case when encoding froman I-frame or the changed regions of the current frame may rely on theimage information from the previous frame and be encoded using aP-frame.

Thus, when the position values of the changed regions of the currentframe matches the position values of the regions in the previous frame,the encoding can be fully specified using one or more I-frames, or theencoding rely on information from the previous frame using one or moreP-frames. However, if the position value for any changed region does notmatch any of the position values from the regions of the previous frame,the changed regions of the current frame will be encoded using one ormore I-frames, as the previous frame does not contain image informationthat corresponds to each region of the current frame. Accordingly, ifthe position values calculated for the changed region of the currentframe are the same as the position values for regions of the previousframe, process flow continues to block 608 where the changed regions ofthe frame are encoded using I-frames or P-frames, as discussed above. Ifthe position values calculated for the changed region are not the sameas the previous changed region, process flow continues to block 614.

At block 614, the changed regions of the frame are encoded usingI-frames. At block 616, it is determined if each frame with changedregions has been encoded. If each frame with changed regions has notbeen encoded, process flow returns to block 612. If each frame withchanged regions has been encoded, process flow continues to block 618.

At block 618, the encoded frames are combined with the related audio. Atblock 620, the encoded frames, related audio, and changed regionposition values are transmitted to a remote adapter. In embodiments, theencoded frames may be encrypted. For example, certain DVDs may includesecurity features entail encrypting the media encoding before it istransmitted. The video encoding from the DVD is encrypted beforetransmission, otherwise, the transmission could fail. At block 622, theencoded frames and audio are decoded as discussed above. In embodiments,the encoded frames are decrypted. At block 624 it is determined if thereis a changed region in the decoded frames. A changed region may bedetected by the presence of changed region position values. If there isa changed region, process flow continues to block 626. If there is nochanged region, process flow continues to block 628.

At block 626, the encoded frames are composed with the static desktopusing the changed region position values as discussed above. At block628, the encoded frames and static desktop may be rendered on a remotedevice.

The process flow diagrams of FIGS. 5 and 6 are not intended to indicatethat the blocks of methods 500 and 600 are to be executed in anyparticular order, or that all of the blocks are to be included in everycase. Further, any number of additional blocks may be included withinthe methods 500 and 600, depending on the details of the specificimplementation.

FIG. 7 is a block diagram showing tangible, non-transitorycomputer-readable media 700 that stores code for media encoding usingchanged regions, in accordance with embodiments. The tangible,non-transitory computer-readable media 700 may be accessed by aprocessor 702 over a computer bus 704. Furthermore, the tangible,non-transitory computer-readable media 700 may include code configuredto direct the processor 702 to perform the methods described herein.

The various software components discussed herein may be stored on one ormore tangible, non-transitory computer-readable media 700, as indicatedin FIG. 7. For example, a managing module 706 may be configured todetect the changed regions in the media encoding. An encoding module 708may be configured to encode the changed regions. Further, a transmittingmodule 710 may be configured to send the changed regions with theirchanged region positions to a remote adapter. A decoding module 712 maybe configured to decode the changed regions. A composition module 714may be configured to compose the changed regions with the static portionusing the changed region position values.

The block diagram of FIG. 7 is not intended to indicate that thetangible, non-transitory computer-readable media 700 is to include allof the components shown in FIG. 7. Further, the tangible, non-transitorycomputer-readable media 700 may include any number of additionalcomponents not shown in FIG. 7, depending on the details of the specificimplementation.

FIG. 8 is a block diagram of an exemplary system 800 for implementingshared physical memory. Like numbered items are as described withrespect to FIGS. 2, 3, and 4. In some embodiments, the system 800 is amedia system. In addition, the system 800 may be incorporated into apersonal computer (PC), laptop computer, ultra-laptop computer, tablet,touch pad, portable computer, handheld computer, palmtop computer,personal digital assistant (PDA), cellular telephone, combinationcellular telephone/PDA, television, smart device (e.g., smart phone,smart tablet or smart television), mobile internet device (MID),messaging device, data communication device, or the like.

In various embodiments, the system 800 comprises a platform 802 coupledto a display 804. The platform 802 may receive content from a contentdevice, such as content services device(s) 806 or content deliverydevice(s) 808, or other similar content sources. A navigation controller810 including one or more navigation features may be used to interactwith, for example, the platform 802 and/or the display 804. Each ofthese components is described in more detail below.

The platform 802 may include any combination of a chipset 812, a centralprocessing unit (CPU) 202, a memory device 204, a storage device 232, agraphics subsystem 814, applications 816, and a radio 818. The chipset812 may provide intercommunication among the CPU 102, the memory device204, the storage device 120, the graphics subsystem 814, theapplications 816, and the radio 818. For example, the chipset 812 mayinclude a storage adapter (not shown) capable of providingintercommunication with the storage device 232.

The CPU 102 may be implemented as Complex Instruction Set Computer(CISC) or Reduced Instruction Set Computer (RISC) processors, x86instruction set compatible processors, multi-core, or any othermicroprocessor or central processing unit (CPU). In some embodiments,the CPU 102 includes dual-core processor(s), dual-core mobileprocessor(s), or the like.

The memory device 204 may be implemented as a volatile memory devicesuch as, but not limited to, a Random Access Memory (RAM), DynamicRandom Access Memory (DRAM), or Static RAM (SRAM). The storage device232 may be implemented as a non-volatile storage device such as, but notlimited to, a magnetic disk drive, optical disk drive, tape drive, aninternal storage device, an attached storage device, flash memory,battery backed-up SDRAM (synchronous DRAM), and/or a network accessiblestorage device. In some embodiments, the storage device 232 includestechnology to increase the storage performance enhanced protection forvaluable digital media when multiple hard drives are included, forexample.

The graphics subsystem 814 may perform processing of images such asstill or video for display. The graphics subsystem 814 may include agraphics processing unit (GPU), such as the GPU 206 (FIG. 2), or avisual processing unit (VPU), for example. An analog or digitalinterface may be used to communicatively couple the graphics subsystem814 and the display 804. For example, the interface may be any of aHigh-Definition Multimedia Interface, DisplayPort, wireless HDMI, and/orwireless HD compliant techniques. The graphics subsystem 814 may beintegrated into the CPU 102 or the chipset 812. Alternatively, thegraphics subsystem 814 may be a stand-alone card communicatively coupledto the chipset 812.

The graphics and/or video processing techniques described herein may beimplemented in various hardware architectures. For example, graphicsand/or video functionality may be integrated within the chipset 812.Alternatively, a discrete graphics and/or video processor may be used.As still another embodiment, the graphics and/or video functions may beimplemented by a general purpose processor, including a multi-coreprocessor. In a further embodiment, the functions may be implemented ina consumer electronics device.

The radio 818 may include one or more radios capable of transmitting andreceiving signals using various suitable wireless communicationstechniques. Such techniques may involve communications across one ormore wireless networks. Exemplary wireless networks include wirelesslocal area networks (WLANs), wireless personal area networks (WPANs),wireless metropolitan area network (WMANs), cellular networks, satellitenetworks, or the like. In communicating across such networks, the radio818 may operate in accordance with one or more applicable standards inany version.

The display 804 may include any television type monitor or display. Forexample, the display 804 may include a computer display screen, touchscreen display, video monitor, television, or the like. The display 804may be digital and/or analog. In some embodiments, the display 804 is aholographic display. Also, the display 804 may be a transparent surfacethat may receive a visual projection. Such projections may conveyvarious forms of information, images, objects, or the like. For example,such projections may be a visual overlay for a mobile augmented reality(MAR) application. Under the control of one or more applications 816,the platform 802 may display a user interface 820 on the display 804.

The content services device(s) 806 may be hosted by any national,international, or independent service and, thus, may be accessible tothe platform 802 via the Internet, for example. The content servicesdevice(s) 806 may be coupled to the platform 802 and/or to the display804. The platform 802 and/or the content services device(s) 806 may becoupled to a network 246 to communicate (e.g., send and/or receive)media information to and from the network 246. The content deliverydevice(s) 808 also may be coupled to the platform 802 and/or to thedisplay 804.

The content services device(s) 806 may include a cable television box,personal computer, network, telephone, or Internet-enabled devicecapable of delivering digital information. In addition, the contentservices device(s) 806 may include any other similar devices capable ofunidirectionally or bidirectionally communicating content betweencontent providers and the platform 802 or the display 804, via thenetwork 246 or directly. It will be appreciated that the content may becommunicated unidirectionally and/or bidirectionally to and from any oneof the components in the system 800 and a content provider via thenetwork 246. Examples of content may include any media informationincluding, for example, video, music, medical and gaming information,and so forth.

The content services device(s) 806 may receive content such as cabletelevision programming including media information, digital information,or other content. Examples of content providers may include any cable orsatellite television or radio or Internet content providers, amongothers.

In some embodiments, the platform 802 receives control signals from thenavigation controller 810, which includes one or more navigationfeatures. The navigation features of the navigation controller 810 maybe used to interact with the user interface 820, for example. Thenavigation controller 810 may be a pointing device that may be acomputer hardware component (specifically human interface device) thatallows a user to input spatial (e.g., continuous and multi-dimensional)data into a computer. Many systems such as graphical user interfaces(GUI), and televisions and monitors allow the user to control andprovide data to the computer or television using physical gestures.Physical gestures include but are not limited to facial expressions,facial movements, movement of various limbs, body movements, bodylanguage or any combination thereof. Such physical gestures can berecognized and translated into commands or instructions.

Movements of the navigation features of the navigation controller 810may be echoed on the display 804 by movements of a pointer, cursor,focus ring, or other visual indicators displayed on the display 804. Forexample, under the control of the applications 816, the navigationfeatures located on the navigation controller 810 may be mapped tovirtual navigation features displayed on the user interface 820. In someembodiments, the navigation controller 810 may not be a separatecomponent but, rather, may be integrated into the platform 802 and/orthe display 804.

The system 800 may include drivers (not shown) that include technologyto enable users to instantly turn on and off the platform 802 with thetouch of a button after initial boot-up, when enabled, for example.Program logic may allow the platform 802 to stream content to mediaadaptors or other content services device(s) 806 or content deliverydevice(s) 808 when the platform is turned “off.” In addition, thechipset 812 may include hardware and/or software support for 5.1surround sound audio and/or high definition 9.1 surround sound audio,for example. The drivers may include a graphics driver for integratedgraphics platforms. In some embodiments, the graphics driver includes aperipheral component interconnect express (PCIe) graphics card.

In various embodiments, any one or more of the components shown in thesystem 800 may be integrated. For example, the platform 802 and thecontent services device(s) 806 may be integrated; the platform 802 andthe content delivery device(s) 808 may be integrated; or the platform802, the content services device(s) 806, and the content deliverydevice(s) 808 may be integrated. In some embodiments, the platform 802and the display 804 are an integrated unit. The display 804 and thecontent service device(s) 806 may be integrated, or the display 804 andthe content delivery device(s) 808 may be integrated, for example.

The system 800 may be implemented as a wireless system or a wiredsystem. When implemented as a wireless system, the system 800 mayinclude components and interfaces suitable for communicating over awireless shared media, such as one or more antennas, transmitters,receivers, transceivers, amplifiers, filters, control logic, and soforth. An example of wireless shared media may include portions of awireless spectrum, such as the RF spectrum. When implemented as a wiredsystem, the system 800 may include components and interfaces suitablefor communicating over wired communications media, such as input/output(I/O) adapters, physical connectors to connect the I/O adapter with acorresponding wired communications medium, a network interface card(NIC), disc controller, video controller, audio controller, or the like.Examples of wired communications media may include a wire, cable, metalleads, printed circuit board (PCB), backplane, switch fabric,semiconductor material, twisted-pair wire, co-axial cable, fiber optics,or the like.

The platform 802 may establish one or more logical or physical channelsto communicate information. The information may include mediainformation and control information. Media information may refer to anydata representing content meant for a user. Examples of content mayinclude, for example, data from a voice conversation, videoconference,streaming video, electronic mail (email) message, voice mail message,alphanumeric symbols, graphics, image, video, text, and the like. Datafrom a voice conversation may be, for example, speech information,silence periods, background noise, comfort noise, tones, and the like.Control information may refer to any data representing commands,instructions or control words meant for an automated system. Forexample, control information may be used to route media informationthrough a system, or instruct a node to process the media information ina predetermined manner. The embodiments, however, are not limited to theelements or the context shown or described in FIG. 8.

FIG. 9 is a schematic of a small form factor device 900 in which thesystem 800 of FIG. 8 may be embodied. Like numbered items are asdescribed with respect to FIG. 8. In some embodiments, for example, thedevice 900 is implemented as a mobile computing device having wirelesscapabilities. A mobile computing device may refer to any device having aprocessing system and a mobile power source or supply, such as one ormore batteries, for example.

As described above, examples of a mobile computing device may include apersonal computer (PC), laptop computer, ultra-laptop computer, tablet,touch pad, portable computer, handheld computer, palmtop computer,personal digital assistant (PDA), cellular telephone, combinationcellular telephone/PDA, television, smart device (e.g., smart phone,smart tablet or smart television), mobile internet device (MID),messaging device, data communication device, and the like.

An example of a mobile computing device may also include a computer thatis arranged to be worn by a person, such as a wrist computer, fingercomputer, ring computer, eyeglass computer, belt-clip computer, arm-bandcomputer, shoe computer, clothing computer, or any other suitable typeof wearable computer. For example, the mobile computing device may beimplemented as a smart phone capable of executing computer applications,as well as voice communications and/or data communications. Althoughsome embodiments may be described with a mobile computing deviceimplemented as a smart phone by way of example, it may be appreciatedthat other embodiments may be implemented using other wireless mobilecomputing devices as well.

As shown in FIG. 9, the device 900 may include a housing 902, a display904, an input/output (I/O) device 906, and an antenna 908. The device900 may also include navigation features 910. The display 904 mayinclude any suitable display unit for displaying information appropriatefor a mobile computing device. The I/O device 906 may include anysuitable I/O device for entering information into a mobile computingdevice. For example, the I/O device 906 may include an alphanumerickeyboard, a numeric keypad, a touch pad, input keys, buttons, switches,rocker switches, microphones, speakers, a voice recognition device andsoftware, or the like. Information may also be entered into the device900 by way of microphone. Such information may be digitized by a voicerecognition device.

EXAMPLE 1

A method for media encoding using changed regions is described herein.The method includes detecting any changed regions and a static portionof a desktop. The changed regions may be encoded and sent to acomposition engine along with their changed region position values. Thechanged regions may be decoded and composed with the static portionusing the changed region position values.

The changed regions may be sent with the changed region position valuesto a composition engine located on a remote adapter. Additionally, thechanged regions may be encrypted and decrypted. Further, the compositionengine may be located on the computing device where the changed regionsand the static portion of the desktop are detected. In embodiments, thechanged regions may be detected by checking a back buffer, and thechanged regions may be sent with the changed region positions to aremote adapter.

The encoding may include encoding from the I-frame if the changed regionposition values do not match previous changed region position values.Further, the encoding may include encoding from the I-frame or theP-frame if the changed region position values match previous changedregion position values. The method may also include iterativelyencoding, decoding, and composing multiple changed regions.Additionally, a media application may use polling, pushing, pulling, ora timer to compose image information.

EXAMPLE 2

A computing device is described herein. The computing device includes acentral processing unit (CPU) that is configured to execute storedinstructions and a storage device that stores instructions. The storagedevice includes processor executable code that, when executed by theCPU, is configured to detect any regions and a static portion of adesktop of the computing device. The changed regions may be encoded.Additionally, the changed regions and their position values may be sentto a composition engine. The changed regions may be decoded and composedwith the static portion using the changed region position values.

The processor executable code may be configured to encrypt and decryptthe changed regions. The composition engine may be located on a remoteadapter, and the changed regions can be decoded at the remote adapter.The computing device may be configured to detect the changed regions bychecking a back buffer, and the changed regions may be sent with thechanged region position values to a remote adapter. The computing devicemay also be configured to encode from the I-frame if the changed regionposition values do not match previous changed region position values.Further, the computing device may be configured to encode from theI-frame or the P-frame if the changed region position values matchprevious changed region position values. The computing device may beconfigured to iteratively encode, decode, and compose multiple changedregions. Additionally, the computing device may be configured to poll,push, pull, or use a timer to compose image information from a mediaapplication. The computing device may also include radio and a display,and the radio and display may be communicatively coupled at least to thecentral processing unit.

EXAMPLE 3

At least one non-transitory machine readable medium having instructionsstored therein is described herein. In response to being executed on acomputing device, the instructions cause the computing device to detectany changed regions and a static portion of a desktop of the computingdevice. The changed regions may be encoded. The instructions may alsocause the computing device to send the changed regions with theirchanged region position values to a composition engine. The changedregions may be decoded and composed with the static portion of thedesktop using the changed region position values.

The changed regions and their changed region position values may be sentto the composition engine, and the composition engine may be located ona remote adapter. The changed regions may be decoded at the remoteadapter. Further, the changed regions are encrypted and decrypted.

It is to be understood that specifics in the aforementioned examples maybe used anywhere in one or more embodiments. For instance, all optionalfeatures of the computing device described above may also be implementedwith respect to either of the methods or the computer-readable mediumdescribed herein. Furthermore, although flow diagrams and/or statediagrams may have been used herein to describe embodiments, theinventions are not limited to those diagrams or to correspondingdescriptions herein. For example, flow need not move through eachillustrated box or state or in exactly the same order as illustrated anddescribed herein.

The inventions are not restricted to the particular details listedherein. Indeed, those skilled in the art having the benefit of thisdisclosure will appreciate that many other variations from the foregoingdescription and drawings may be made within the scope of the presentinventions. Accordingly, it is the following claims including anyamendments thereto that define the scope of the inventions.

What is claimed is:
 1. A method for media encoding using dirty regions,comprising: determining a number of dirty regions, wherein a change in apre-determined number of pixels is to indicate a dirty region; writingthe dirty regions to a buffer; encoding the dirty regions in response tothe number of dirty regions being greater than a threshold number,without encoding a desktop surface.
 2. The method of claim 1,comprising: sending the dirty regions and dirty region position valuesto a composition engine; decoding the dirty regions; and composing thedirty regions with the desktop surface using the dirty region positionvalues.
 3. The method of claim 1, wherein each dirty region is detectedwhen a number of changed pixels within a region is greater than apredetermined threshold.
 4. The method of claim 1, comprising: sendingthe dirty regions and any dirty region position values to a compositionengine, wherein the composition engine is located on a remote adapter;and decoding the dirty regions at the remote adapter.
 5. The method ofclaim 1, wherein a composition engine is located on the computing devicewhere the dirty regions are detected.
 6. The method of claim 1, furthercomprising: detecting the dirty regions by checking a back buffer;sending the dirty regions with any dirty region positions to a remoteadapter.
 7. The method of claim 1, comprising encoding from an I-framein response to dirty region position values not matching previous dirtyregion position values.
 8. The method of claim 1, comprising encodingfrom an I-frame or a P-frame in response to dirty region position valuesmatching previous dirty region position values.
 9. The method of claim1, comprising iteratively detecting, determining, and encoding dirtyregions.
 10. A computing device, comprising: a central processing unit(CPU) that is configured to execute stored instructions and a storagedevice that stores instructions, wherein the storage device includesprocessor executable code that, when executed by the CPU, is configuredto: determine a number of dirty regions, wherein a change in apre-determined number of pixels is to indicate a dirty region; and anencoder configured to encode the dirty regions in response to the numberof dirty regions being greater than a threshold number, without encodinga desktop surface.
 11. The computing device of claim 10, wherein theprocessor executable code, when executed by the CPU, is configured to:send the dirty regions and dirty region position values to a compositionengine; decode the dirty regions; and compose the dirty regions with thedesktop surface using the dirty region position values.
 12. Thecomputing device of claim 10, wherein each dirty region is detected whena number of changed pixels within a region is greater than apredetermined threshold.
 13. The computing device of claim 10, whereinthe processor executable code, when executed by the CPU, is configuredto: send the dirty regions and any dirty region position values to acomposition engine, wherein the composition engine is located on aremote adapter; and decode the dirty regions at the remote adapter. 14.The computing device of claim 10, further comprising a compositionengine.
 15. The computing device of claim 10, wherein the computingdevice is configured to: detect the dirty regions by checking a backbuffer; send the dirty regions with any dirty region position values toa remote adapter.
 16. The computing device of claim 10, wherein thecomputing device is configured to encode from an I-frame in response todirty region position values not matching previous dirty region positionvalues.
 17. The computing device of claim 10, wherein the computingdevice is configured to encode from an I-frame or a P-frame in responseto the dirty region position values matching previous dirty regionposition values.
 18. The computing device of claim 10, wherein thecomputing device is configured to iteratively detect, determine, andencode dirty regions.
 19. The computing device of claim 10, wherein thecomputing device is configured to poll, push, pull, or use a timer tocompose image information from a media application.
 20. The computingdevice of claim 10, further comprising a radio and a display, the radioand display communicatively coupled at least to the central processingunit.
 21. At least one non-transitory machine readable medium havinginstructions stored therein that, in response to being executed on acomputing device, cause the computing device to: determine a number ofdirty regions, wherein a change in a pre-determined number of pixels isto indicate a dirty region; and encode the dirty regions in response tothe number of dirty regions being greater than a threshold number,without encoding a desktop surface.
 22. The at least one non-transitorymachine readable medium of claim 21, having instructions stored thereinthat, in response to being executed on a computing device, cause thecomputing device to: send the dirty regions and dirty region positionvalues to a composition engine; decode the dirty regions; and composethe dirty regions with a static portion of the desktop using the dirtyregion position values.
 23. The at least one non-transitory machinereadable medium of claim 21, wherein each dirty region is detected whenthe number of changed pixels within a region is greater than apredetermined threshold.